A memory-based NFA regular expression match engine for signature-based intrusion detection

Signature-based intrusion detection is required to inspect network traffic at wire-speed. Matching packet payloads against patterns specified with regular expression is a computation intensive task. Hence, the design of hardware accelerator to speed up regular expression matching has been an active research area. A systematic approach to detect regular expression is based on finite automaton. The space-time trade-off between deterministic finite automaton (DFA) and non-deterministic finite automaton (NFA) is well-known. DFA can offer constant throughput but it may suffer from the state explosion problem. Hence, implementation of DFA for large pattern sets on embedded device with limited on-chip memory may not be viable. NFA requires linear space but the throughput can be very low. Implementations of NFA with hardwired circuits can overcome the speed deficiency by exploiting the massive parallelism offered by dedicated hardware circuitries, but this approach does not support efficient dynamic updates. In this paper, we shall present a memory-based architecture for the implementation of NFA to speed up regular expression matching for signature-based intrusion detection. The proposed method supports dynamic updates and offers constant throughput so that it can be used to supplement the existing DFA-based methods in handling large pattern sets. Two categories of intrusion detection techniques are used in today's intrusion detection system (IDS), namely anomaly detection and signature-based detection. Anomaly detection methods [1] detect attacks by monitoring network traffic behaviors. If the observed traffic behavior deviates significantly from the expect user profile, the IDS may generate an alert and/or take appropriate actions. Anomaly detection can be used to protect the system against unknown attacks. The detection rate is about 98% while the false alarm rate is about 1% [2]. Anomaly detection has two limitations. First, by the time the attack is detected some damages might have been done to the computer system. Anomaly detection can reduce and localize the damages. Second, anomaly detection may not be able to detect slow-attacks where the attacker deliberately slows down the traffic rate of the attack to avoid detection. Signature-based detection methods are based on content inspection. Signature-based detection is precise and accurate, but it has one limitation. The attack patterns have to be known in advance , and are characterized by the intrusion signatures. If an intrusion signature is found in the packet header/payload, the IDS will execute the predefined action, e.g., generates an alert and/or blocks the packet. Pattern matching is a computation intensive task. In …